On Flexible Allocation of Index and Temporary Data in Parallel Database Systems

Data placement is a key factor for high performance database systems. This is particularly true for parallel database systems where data allocation must support both I/O parallelism and processing parallelism within complex queries and between independent queries and transactions. Determining an effective data placement is a complex administration problem depending on many parameters including system architecture, database and workload characteristics, hardware configuration, etc. Research and tool support has so far concentrated on data placement for base tables, especially for Shared Nothing (SN), e.g. [MD97]. On the other hand, to our knowledge, data placement issues for architectures where multiple DBMS instances share access to the same disks (Shared Disk, Shared Everything, specific hybrid architectures) have not yet been investigated in a systematic way. Furthermore, little work has been published on effective disk allocation of index structures and temporary data (e.g., intermediate query results). However, these allocation problems gain increasing importance, e.g. in order to effectively utilize parallel database systems for decision support / data warehousing environments. In the next section we discuss the index allocation problem in more detail and introduce a classification of various approaches that are already supported to some degree in commercial DBMS. While SN offers only few options, the other architectures provide a higher flexibility because index allocation can be independent from the base table allocation. For certain indexsupported queries, this can allow for order-of-magnitude savings in I/O and communication cost. We then turn to the disk allocation of intermediate query results for which the allocation parameters can be chosen dynamically at query run time. For the case of parallel hash joins, we outline how to determine an optimal approach supporting a high degree of parallelism. The work discussed is performed within a project aiming at developing strategies to automatically determine optimal data allocation strategies in order to simplify system administration in high performance environments.